Multidisciplinary Programed Learning Simulation to Improve Visual Blood Loss Estimation for Obstetric Trauma Scenarios

Introduction: We designed and implemented a Programmed Learning Simulation (PLS) exercise depicting obstetric scenarios of hemorrhage to train anesthesiologists, ancillary staff, and surgeons to accurately estimate blood loss visually. We then measured the efficacy of this exercise in a clinical setting. Methods: We conducted a prospective study to assess the effect of implementing a PLS exercise on quantification of blood loss in an operative setting. The PLS exercise consisted of 13 simulation stations of varying quantities of simulated blood loss paired with standardized objects of known volume. Eighty-eight individuals participated including attending physicians, residents, medical students, and ancillary staff participated in this study. The PLS was part of regularly scheduled continuing medical education activities; thus, the sampling used was non-randomized convenience method. The percent error was calculated for each of the 13 stations. A subgroup analysis was performed to assess the effect of the years of experience, size of hemorrhage, and occupation on accuracy. Univariate analyses for continuous variables were compared using a one-way ANOVA test. For the comparison of the three groups (years of experience and size of hemorrhage), a p-value of <0.02 was considered statistically significant and for 5-way comparison (professional grouping) a p <0.01 was considered significant after application of the Bonferroni correction (α=0.05). (Part A). To determine the effect of PLS in a clinical setting, the percent error of blood loss estimation for cesarean deliveries during the two-month period after the PLS exercise was compared to the two-month period immediately prior to using the student’s t-test with p<0.05 as significant (Part B). Statistical analysis was performed using International Business Machine, Statistical Package for the Social Sciences, Version 26.0 (IBM SPSS). Results: During Part A, the baseline performance of the participants was evaluated during the PLS activity. The PLS data showed no significant difference in absolute value of mean percent error estimation (standard deviation) across professions: student 63.61% (69.74), ob/gyn 56.91% (47.72), ancillary 62.15% (77.90), general/trauma surgeon 66.70% (65.06), anesthesia 61.51% (63.12). (p = 0.681), or levels of experience 0-5: 62.21% (60.06), 6-10 years: 56.22% (52.66), greater than 10 years: 61.89% (71.89) (p = 0.831). However, mean percent error of estimation was higher when participants estimated smaller samples 77.7% (104.73) compared to either medium 56.8% (49.06) or large 57.9% (46.19) samples (p<0.001). For Part B, 179 cesarean deliveries occurred during the pre-intervention period and 193 occurred during the post-intervention period. Mean error in provider estimation of blood loss significantly improved from 47% (68.51) pre-intervention to 31% (32.70) post-intervention (p=0.009). Conclusion: We believe our described PLS activity was effective in teaching techniques for visual blood loss estimation. This was reflected by improved competency in a clinical setting, demonstrated by more accurate visually estimated blood loss during the period immediately following simulation activity compared to a prior time frame. Further research is needed to assess the impact of simulation activities on patient outcomes, such as utilization of blood products and patient morbidity.


Introduction
T rauma complicates 6-7% of all pregnancies and is the leading cause of non-obstetric maternal morbidity and mortality. An analysis conducted by WHO reported about 73% of all maternal mortalities worldwide were attributed to true obstetric etiologies with hemorrhage as the leading cause of death (1). Identification of hypovolemic shock during pregnancy poses challenges due to the unique physiological adaptions, including increased plasma volume, stroke volume, and cardiac output (2). Shock may not manifest until the total blood volume depletion reaches 30-35%, as compared to 15% in a non-pregnant patient (3)(4)(5). Clinicians are unable to rely on vital signs, physiological changes, or indications of end organ damage until patients are in a critical condition. The use of vital signs alone is impractical. Thus, accurate estimation of cumulative blood loss can forewarn impending hemodynamic compromise.
Estimation methods are non-standardized, but a few commonly used techniques include visual, gravimetric, and colorimetric methods. The visual method involves estimation by the clinician through visual assessment of blood contaminated surgical equipment like surgical sponges and suction containers. Gravimetric methods involve subtracting blood-soaked surgical equipment from their dry weights to indirectly measure the blood loss. This value is then combined with the amount of liquid present in collection containers (6). Lastly, colorimetric estimation uses technology platforms to calculate the blood loss from photographs of soaked surgical materials. The algorithm filters out nonblood, colored components in the photographs through colorimetric analysis and determines hemoglobin mass present (6,7). Clinically, during simulation, quantitative methods like gravimetric and colorimetric have been shown to improve blood loss estimation (7)(8)(9)(10)(11)(12). For instance, Lilley et al. showed gravimetric estimation of blood loss offered a significantly lower mean percent error (4.0±2.7%) in estimation compared to visual estimation (34.7±32.1%) (12). Gerdessen et al. performed a meta-analysis which compared modalities of blood loss estimation and found that colorimetric techniques offered the highest degree of accuracy in blood loss estimation. This is supported by another study which reported visual and gravimetric methods having a higher degree of bias when compared to colorimetric ones (7). Medical literature attempts to prove one modality superior to another; however, these studies show conflicting results and state the evidence to be insufficient (13). In addition, each method possesses its own flaws for measurement and is vulnerable to inaccuracy (7). For instance, the addition of amniotic fluid in obstetric procedures as well as external blood loss in trauma situations can confound gravimetric analysis. A major barrier to implementation of colorimetric analysis is access to an artificialintelligence technology.
Historically, visual estimation is the most practical method of measurement, especially in obstetric emergencies, where quantitative measures cannot feasibly be applied due to time and physical constraints (13)(14)(15). However, the inaccuracy of visual blood loss assessment has been illustrated in several studies (8,(16)(17)(18). Stafford et al. compared the accuracy of visually estimated blood loss (EBL) to calculate blood loss (CBL) for both vaginal and cesarean deliveries. The results found EBL to be significantly inaccurate when compared to CBL for both modes of delivery (18). These findings remain true in a recent simulation study that showed inaccuracies in visual EBL in a video series of emergent and non-emergent to either medium 56.8% (49.06) or large 57.9% (46.19) samples (P<0.001). For Part B, 179 cesarean deliveries occurred during the pre-intervention period and 193 occurred during the postintervention period. Mean error in provider estimation of blood loss significantly improved from 47% (68.51) pre-intervention to 31% (32.70) post-intervention (P=0.009). Conclusion: We believe our described PLS activity was effective in teaching techniques for visual blood loss estimation. This was reflected by improved competency in a clinical setting, demonstrated by more accurate visually estimated blood loss during the period immediately following simulation activity compared to a prior time frame. Further research is needed to assess the impact of simulation activities on patient outcomes, such as utilization of blood products and patient morbidity. Keywords: Simulation learning; Cesarean section; Obstetric injuries (19). Most studies have shown that at lower volumes visual estimation has a tendency to overestimate blood loss (8,19). On the other end, at higher volumes, visual estimation tends to underestimate blood loss (4,20). Supporting this tendency of underestimation, Lertbunnaphong et al. found visual techniques underestimated blood loss compared to the gravimetric drape method, as the visual method missed 65.4% of post-partum hemorrhage diagnoses (20). Another supporting study reported that the participants overestimated smaller blood volumes between 50ml to 200ml and underestimated blood volumes greater than 400ml. This study also reported a greater percent error in estimation as blood volumes increased (19). One study disapproved the inaccuracy of the visual method and showed it to be equal to CBL when approximating the volumes <500mL (21). Additional factors such as professional experience and training show conflicting results, with some showing no improvement in accuracy (10,22,23). Regardless of these shortcomings, the decision to transfuse in obstetric emergencies has shown no significant difference between various modalities of blood loss estimation (9,24,25).
Our simulation is novel as it utilizes a programmed learning didactic approach. Programmed Learning/Instruction is the process of arranging the material to be learned into a series of sequential steps to help the learner form mental associations between familial and unfamiliar concepts. As an educational technique, the learner is presented a logical sequence of materials, with multiple content repetitions. Immediate feedback is given, which serves as reinforcement of the content.
A systematized method of blood loss estimation may increase awareness that a massive obstetric hemorrhage has occurred, thereby allowing for earlier intervention and improved communications during trauma situations and transition of care. Thus, the objective of this study was to design and implement a simulation exercise depicting obstetric scenarios of hemorrhage to train anesthesiologists, ancillary staff, and surgeons to estimate blood loss visually more accurately. We hypothesized that a programmed learning didactic simulation exercise would improve the provider's estimation of blood loss in a clinical setting. In order to test this hypothesis, we compared the percent error of blood loss estimation for cesarean deliveries after the simulation exercise to the period immediately prior.

Part A: The Programmed Learning Simulation
We conducted a prospective study to assess the effect of implementing a PLS exercise on quantification of blood loss in an operation setting. This study was reviewed by the Institutional Review Board of our affiliated university, New York Medical College, Valhalla, New York, and received an exemption. The PLS was part of regularly scheduled continuing medical education activities, thus the sampling was a non-randomized convenience method. The programmed learning simulation (PLS) exercise was administered to 88 learners consisting of 24 third-year medical students, 19 obstetrics and gynecology physicians, 22 ancillary staff members (operation room nurses and technicians), 10 trauma surgeons, and 14 anesthesiologists at Richmond University Medical Center, Staten Island, New York. For the residents and attending physicians, participation was mandatory as part of their continuing medical education. Nurses, ancillary staff, and medical students were highly encouraged to participate and did so voluntarily. The participants were told that their answers may be used for quality improvement/research purposes; however, there was no formal consent process.
The PLS was conducted jointly by attending physician leaders in the ob/gyn and trauma surgery departments. The didactic activity took place over the course of three sessions spanning a ten-day time period. For each session, the stations were recreated with fresh artificial blood to avoid spoilage. Participants were expected to maintain confidentiality and requested not to share information regarding the simulations or answers to the simulation station estimates with their colleagues.
The PLS exercise was designed to teach the providers how to visually quantify blood loss using a series of thirteen simulation stations. Each station depicted a simulated amount of blood loss for a clinical scenario. For this PLS exercise, artificial blood was prepared using nonvalidated mixture light corn syrup, water, and red food coloring with the first two ingredients in a 5:1 ratio and drops of coloring added until the desired hue was achieved. However, several more contemporary recipes are available based on the desired quantity and consistency (26). All operating room supplies were obtained from the labor and delivery floor of Richmond University Medical Center.
The ninety-minute PLS exercise was executed in two parts, followed by a debriefing. For Part 1 (twenty-six minutes), the participants visited each of the thirteen stations (two minutes each) and were asked to visually quantify the amount of simulated blood loss depicted. They solely used visual cues such as the number of soaked laps, appearance of operation field, etc.; however, they were not permitted to touch or move any of the objects. The thirteen stations were arranged in a circuit design. Two to three participants were positioned at each of the thirteen stations along the circuit. The participants were instructed to work individually and not to communicate the answers with each other. After the twominute period elapsed, an alarm was sounded, and participants were instructed to move to the next station along the circuit loop, while at each station, participants documented their answers on a response sheet that was collected at the end of Part 1. The response sheet solicited demographic information (department, level of experience) and had thirteen blank numbered lines for free form documentation of answers to each simulation station. During Part 1 of the PLS activity, each simulation station was viewed without its corresponding paired object.
Following Part 1 and after collection of the response sheets, paired objects along with a card revealing the correct answer were placed at each station. This commenced the transition to Part 2 (thirteen minutes) where participants, in groups or two-three people, revisited each of the thirteen stations for an additional one minute. Participants were free to discuss the station, paired objectives, and original answers among the group members at this time. Like Part 1, an alarm bell was sounded to signal when it was time to move to the next station along the circuit. The final debriefing session (approximately fifty minutes) consisted of open discussion and didactic teaching session. The debriefing was conducted by physician leaders in the ob/gyn department and trauma surgery departments who facilitated the simulation. During this session, the answers to each of the thirteen simulations were discussed. Additionally, formal techniques for visually estimating blood loss were presented to the participants (6). Each person's performance was measured by scoring of the response sheet that was collected at the end of Part 1. By providing immediate feedback to the learner during the debriefing session, we were consistent in maintaining the core tenets of programmed learning simulation.

Statistical Analysis for Part A
All statistical analyses were carried out using International Business Machine, Statistical Package for the Social Sciences, Version 26.0 (IBM SPSS). In order to assess normality, data points were plotted and observed. Outliers were not excluded from analysis as these extreme data points reflect trends in gross overestimation or underestimation of blood loss is well documented in the literature (4,8,(19)(20). We utilized both percent error and absolute value of percent error as they account for different aspects of error in estimation. The percent error calculation accounts for bidirectional error in either overestimation or underestimation. However, it does not account for the magnitude of error as the positive values cancel the negative ones. Thus, we also report the absolute value of the percent error as it accounts for the absolute magnitude of error in either direction.
The percent error of estimated blood loss was calculated for each participant response using the Following formula (17): Descriptive statistical analysis of percent error (mean, range) was calculated for each of the thirteen stations. The absolute value of percent error estimation was calculated by taking the absolute value of the percent error calculation. A pooled subgroup analysis for all thirteen stations was performed to assess the influence of years of experience (0-5 years, 6-10 years, and greater than 10 years), size of hemorrhage (Small: 30 ml, 60 ml, and 100 ml; Medium: 150 ml, 250 ml, and 500 ml; Large: 1000 ml, 1500 ml, and 2000 ml), and occupational status (medical student, resident, obstetric surgeon, general/trauma surgeon, anesthesiologist, and ancillary staff-OR nurses or technicians) on the accuracy of estimation. Univariate analyses for continuous variables were compared using oneway ANOVA. In order to minimize the chance of making at Type 1 error when making multiple group comparisons, we applied a Bonferroni correction (α new =α original / number of groups) to an alpha original of 0.05. Thus, for the three group comparisons (years of experience and size of hemorrhage), a p-value of <0.02 (0.05/3) was considered statistically significant and for 5-way comparison (professional grouping) a P<0.01 (0.05/5) was considered significant after application of a Bonferroni correction. When differences between three or more groups were statistically significant, post-hoc analysis was performed to detect a significant differences between the groups.

Part B: Effect of the PLS in a Clinical Setting
In order to test the effect of the PLS activity in a clinical setting, we compared the error of blood loss estimation for cesarean deliveries after the simulation exercise to the period immediately prior. Cesarean deliveries performed during the two-month period prior to the simulation exercise were included in the pre-interventional group and cesarean deliveries that occurred in the two-month period after the simulation exercise were included in the post-interventional group. Exclusion criteria were emergency cesarean deliveries (fetal distress, placental abruption, cord prolapse etc.), patients transfused with blood products intraoperatively or post-operatively during the first 24 hours, or those patients that refused a post-operative Complete Blood Count (CBC). A de-identified database grouped by the month when the cesarean sections occurred was maintained.
At our institution, it is routinely practiced for the estimated blood loss (EBL) for each cesarean delivery to be determined by an informal joint consensus of the attending surgeon, resident, and anesthesiologist. The final documentation EBL is done by the resident in the medical record. This practice was the same both prior to and after the PLS. The staff were not told that their EBL values were being monitored by the obstetric department after the PLS activity to avoid observational bias. The participants in Part B included the 19 ob/ gyn physicians, 22 ancillary staff members, and 14 anesthesiologists who took part in the PLS exercise.
The calculated blood loss (CBL) was obtained by multiplying the maternal blood volume by the percent change in hematocrit (18): Where the maternal blood volume was determined for a standardized formula (18):

Maternal Blood Volume=0.75×[(maternal height in inches ×50)×(maternal weight in pounds ×25)]
and the percent change in hematocrit was calculated from the following formula (17): The pre-delivery CBC, used to assess the hematocrit, was obtained at admission to labor and delivery, prior to the cesarean delivery. The post-operative CBC was performed the morning after the index cesarean delivery, as done routinely at our institution on all post-cesarean patients. Maternal weight and height were determined at admission to Labor and Delivery.
The percent error in EBL was calculated using the CBL as a basis of comparison yielding a percent error value (17):

Statistical Analysis for Part B
Descriptive statistical analysis of percent error (mean, range) was calculated for each pre-and post-interventional period. The absolute value of percent error estimation was calculated by taking the absolute value of the percent error calculation. Comparison between the pre-and post-intervention data sets was analyzed using student's t-test. A P-value <0.05 was considered statistically significant.

PartA
A total of 88 clinicians, consisting of 24 medical students, 19 ob/gyn physicians, 22 ancillary staff members, 10 surgeons, and 14 anesthesiologists participated in the PLS exercise. Years of experience among providers were: 0-5 years (n=39), 6-10 years (n=9), and greater than 10 years (n=40). Table 1 depicts mean percent error and absolute value of mean percent error for all the thirteen stations. The results of each station in Table 1 are presented as a mean aggregate of all learners that participated in the simulation.
The mean and range of percent error, which represent the range of error in over-and underestimation, are shown in the first column of Table 2. The significant p-values indicate that the mean estimates differ from one another; however it does not indicate that one group was more accurate at estimation than another. For example, the medical students mean percent error was -0.44, which seems better than the ob/ gyn physicians who had a mean percent error of -29.95. However, the range of error for medical students was much broader, accounting for the result. Thus, there are limitations to solely relying on the mean percent error calculation. The most notable limitation is that it does not account for the actual magnitude of error as the positive values (overestimation) cancel the negative ones (underestimation). Thus, we also report the absolute value o the mean percent error, a metric which accounts for the magnitude of error regardless of over-or under-estimation.
( Table 2). The box-and-whisker plots in Figures  1-3 visually depict the mean absolute error and percent error in estimation, absolute range of error estimates, interquartile range (blue box), all outliers for the professional groupings (2A), years of experience (2B), and volume sizes (2C).

Part B
There were 179 cesarean procedures (86 primary, 93 repeat) that met the inclusion criteria during the two-month pre-interventional period and 193 (105 primary, 88 repeat) during the twomonth post-intervention period. The providers' errors in blood loss were compared between the two periods.
The mean percent error (standard deviation), which accounts for bidirectional error in either over-estimation or under-estimation, was reduced from 23.4% (79.31) pre-intervention to 10.6% (44.17) post-intervention; however, this reduction just fell short of statistical significance (P=0.053). (Table 3) The absolute value of mean error (standard deviation) in provider-estimated blood loss was significantly reduced in the postinterventional period to 31.5% (32.69), compared to 47.2% (68.51) pre-intervention (P=0.009), indicating a statistically significant improvement in post-intervention accuracy of blood loss estimation. Table 3 shows comparison of pre-and postsimulation calculated blood loss (mean, range), estimated blood loss (mean, range), percent error (mean, range), and absolute value of mean percent error (mean, range).

Discussion
The importance of quantifying the amount of blood loss associated with obstetric hemorrhage should not be undervalued. Due to physiological changes associated with pregnancy, signs of hemodynamic instability such as tachycardia and hypotension do not manifest initially (2,3). Although colorimetric and gravimetric techniques of blood loss estimation may be utilized, weighing of materials and using gravimetric measurements are time consuming and not always practical in clinical and trauma situations (7,(13)(14)(15). By designing a simulation exercise to teach providers techniques in visual estimation of blood loss, we sought to improve recognition of massive blood loss that can occur in intraoperative and obstetrical trauma situations. Early recognition of a massive hemorrhage may improve the patient outcomes by prompting the providers to initiate morbidity-reducing interventions, such as prolonged observation in a recovery room or immediate transfusion of blood products.
Programmed Learning/Instruction is the process of arranging the material to be learned into a series of sequential steps to form the associations between familial and unfamiliar concepts. Programmed Learning is a term derived from the tenants of operant or conditioned learning. It was first proposed by B.F. Skinner of Harvard University in the 1940's in a paper entitled 'Science of Learning and art of Teaching' (27). In this paper, Skinner argued that the desirable behavior can be brought out by continuous feedback to the learner. Our programed learning simulation, inspired by this model of teaching, was designed in two steps. During Part 1, learners were exposed to simulated volumes of blood loss and asked to visually quantify it. During Part 2, familiar objects of known volumes were paired with each simulation station. By using known familiar objects, we sought to program the association between the object and the simulated volume visualized. Although our study was not designed to test this hypothesis, we felt that the use of a concrete object helped our learners quantify abstract volumes. For example, some learners may struggle, if asked, to mentally conceptualize 750ml of blood; however, most can easily picture a bottle of wine, which also contains 750ml of fluid. Our programed learning approach builds upon this concept of association.
Majority of prior studies conducted to determine the impact of didactic training on blood loss assessment have demonstrated a very shortterm increase in accuracy post training. Dildy et al. presented a twenty-minute Power Point presentation where techniques to estimate blood loss were discussed (28). Pre-and post-test using seven clinical reconstructions were administered to fifty-three clinical providers. Median percent error in blood loss estimation was significantly reduced for all seven clinical reconstructions post-didactic session. Similarly, a study by Poolnoi et al. used pictograms of measured blood volumes designed to educate providers on techniques of blood loss estimation. Fortynine providers participated in pre-and post-tests where they were given pictures from cesarean sections and asked to estimate the blood volume depicted. After the didactic activity, the accuracy of estimation increased significantly from 30.9% to 61.8% (29). Sukprasert et al. randomized ninety obstetric nurses into two groups: one receiving a didactic course on blood loss estimation and the other received no instruction (30). The nurses that received the educational session had a significantly greater percent accuracy on estimating blood loss as assessed by a post-test clinical reconstruction compared to the nurses who received no instruction. Unlike many of the previous studies which tested the impact of a didactic session using an immediate post-test as the endpoint, our study assessed the effect of a didactic session in clinical practice over a prolonged two-month period. A study by Toledo et al. also assessed longitudinal retention of learning using web-based simulations of blood loss. At nine-months follow-up, the median percent error in estimation declined from -13.5% immediately post-training to -34.6% (31). Similarly, the Illinois Obstetric Hemorrhage Project mandated all obstetric service providers in the State of Illinois, to present a didactic activity on blood loss estimation consisting of a pre-test, didactic lecture, skill stations, and debriefing.
Pooled data from 95 hospitals (n=9,456) indicated improvement in estimation, with participants scoring above 88% correct answers increasing from 10.9% on the pre-test to 49.1% at 6-month follow-up (32). Although both these studies measure longitudinal retention of learning using a post-test, we used a clinical endpoint to test the effectiveness of our didactic simulation exercise over time. For this clinical endpoint, we decided to use cesarean delivery as our clinical representation of obstetric hemorrhage because it is standardized and has occurred frequently.
This, however, was not without limitations. First, the formula used to derive calculated blood loss relied on the differences between pre-and post-operative hematocrit measurements. Whilst it is a standard practice to collect this information on each patient at our hospital, the formula does not account for post-operative bleeding and fluid replacement. We attempted to compensate for it by excluding patients that required blood transfusion within 24 hours of delivery. Second, on average there was at least one person at each cesarean delivery who attended the PLS training. This was due to logistic reasons (holidays, maternity leave, off-duty hours, etc.) and not every resident and attending physician attended the PLS activity. We suggest that other hospitals implementing similar didactic activities schedule multiple sessions throughout the calendar year to expand attendance. In spite of these limitations, the initial results of this study are encouraging as it demonstrated that our programmed learning simulation exercise improved the providers' accuracy of blood loss estimation, and this improvement persisted for two months after the intervention.
Several studies have shown overestimation at low volumes (8,19) and underestimation a high volume (4,20), both of which can have clinical implications. Whilst underestimation at high volumes can lead to the heavy obstetric hemorrhage being unrecognized, underestimation at low volumes may trigger over transfusion of blood products and increase both patients' stress and provider's anxiety. Interestingly, in our study, we saw a trend towards underestimation at low volumes and overestimation at medium and high blood volumes (mean percent error, Table 2). The largest magnitude of error occurred at small volume blood loss (absolute value percent error, Table 2). Our groupings of small, medium, and large were based upon the chosen volumes of the thirteen simulated stations. For learning purposes, some of the volumes were repeated. We felt presenting the same volume in different simulated scenarios would aid in learning, visualization, and retention as they may appear differently in unique situations. Additionally, given the similarities between the higher volume in the small group (100 ml) and smaller volumes in the medium group (150 ml and 250 ml), some overlaps might have occurred, limiting final conclusions in this area. Outliers were not excluded from analysis as we felt that these data points were legitimate as significant over-and under-estimation of blood loss is well documented in the literature (4,8,19,20).
Although one may assume more experienced clinicians to have improved accuracy compared to novice clinicians, the literature is conflicting. Similar to some studies, we found no difference in accuracy between the provider groups or increasing years of experience (10,22,23). Rothermel and Lipman compared the accuracy of visually estimated blood loss across different specialties, years of experience, and levels of confidence in assessment. The study demonstrated that when presented with operative simulations of blood loss in varying magnitudes, surgeons, anesthesia providers, nurses, and technicians demonstrated similar levels of inaccuracy in visual estimation. There was also no association between increasing years of experience or confidence in ability to quantify blood loss with respect to accuracy of estimation (33). However, Dildy et al. found a higher likelihood of blood loss underestimation in cesarean deliveries in professionals with longer years of experience (28). Given that visual blood loss estimation is an abstract concept, years of clinical practice or professional experience may not be sufficient to improve their skill. Of interest, with regard to provider experience, the "6-10 years of experience" group only had 9 participants, while the other two groups averaged around 40. This was due to the large number of medical students/ residents and senior attendings at our institution. This strengthens the argument that didactic instruction with assessment and feedback is needed for providers at all levels of experience.
Our study did not collect data on clinical outcomes such as time to activation of transfusion protocols and utilization of blood products preand post-PLS. One study that compared the use of quantitative blood loss (QBL) methods to visual EBL in post-partum hemorrhages (PPH) showed similar outcomes for the use of blood transfusion, i.e. 2.7% and 2%, respectively (34). Another study which compared EBL and QBL to predict the need for postpartum blood transfusion also showed no difference with QBL reporting higher rates of blood loss >1000ml (24). The ability to visually estimate blood volumes to guide management, especially when initiating blood saving products, is an important skill in trauma and surgery. Further research on the effect of simulation training on clinical outcomes is necessary.
Strengths of our PLS include use of a programmed teaching approach that paired familiar objects with specified hemorrhage simulations. Additionally, our PLS was administered across multiple specialties and varying skill levels. We chose a clinical end point, cesarean delivery, to test the effect of our PLS and found improvement in estimation. At our institution, it has always been a routine practice for providers to informally discuss blood loss and come to an agreed estimate. Thus, conditions were similar pre-and post-PLS. Moreover, after the PLS was conducted, providers were not told that their estimates were being monitored, eliminating observational bias. However, there were limitations to choosing cesarean delivery as an endpoint as discussed previously. Additional limitations include non-standardized groupings of small, medium, and large objects, and small number of stations utilizing surgical equipment (gauzes, laps, chux, etc.). However, simulation stations can be customized to the group of learners depending on the concepts and skills being taught.

Conclusion
Visual estimation of blood loss is a teachable skill. We believe our described PLS activity was effective in teaching providers techniques for visual blood loss estimation. Our results demonstrated improved competency in a clinical setting, with more accurate visually estimated blood loss immediately following the simulation activity compared to pre-intervention.
Future research relating to visual blood loss estimation is needed to improve clinical practice. Studying the longevity of simulation training will help decide the frequency of simulation labs to ensure retention and improve clinical outcomes. Additionally, further studies are necessary to assess the impact of simulation activities on patient outcomes such as utilization of blood products and patient morbidity.